Historically, computer files (or more generally, digital objects) have been stored in file systems. These file systems have typically been hierarchical, and have allowed files to be inserted, removed or retrieved according to a particular schema. Usually, such a file system is implemented using a B-tree and objects are stored along with metadata such as a file name and other attributes. The file identifier often conforms to a regular hierarchical path and files are stored and retrieved using path names.
This model of storing files, though, is reaching its limits as massive amounts of information are now being required to be stored within file systems. A single computer may store millions of files and computer servers in large networks may be required to store many times that amount of information. While a B-tree implementation (for example) may work fine with many thousands of files, a file system may process requests much more slowly as the number of files increase. New techniques of storing information have accordingly been developed.
For instance, Caringo, Inc. of Austin, Tex. has developed a technique where digital objects are stored in a flat address space across any number of computer nodes in a storage cluster. A unique identifier for each object (such as a random number, for example) is used to add the digital object to, or retrieve it from, the storage cluster. When an object is requested by an application, a unique multicast technique is used to send a message to all nodes within the cluster to find the object. Similarly, storing an object also involves sending a message to all nodes to find a suitable storage location, meaning that the object might be stored on any of the nodes. Although this technique is efficient, it can also be affected by the increase in the amount of information that needs to be stored within a cluster.
Accordingly, improvements are desired in the area of storage and retrieval of objects in storage clusters.